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Researchers  have  long  focussed  on  using  physiological 
measures  as  an  indicator  of  mental  workload.  Research  with  transient  and  steady 
state  evoked  potential  electroencephalograph  (EEC)  signals  has  provided  the  basis 
for  further  study  in  mental  state  estimation.  Studies  involving  sum-of-sines 
steady  state  evoked  potentials  have  demonstrated  a  correlation  between  spectral 
changes  and  changing  cognitive  workload.  It  has  also  been  found  that  subjects  can 
learn  to  control  their  responses  to  steady  state  visual  stimulus,  provided  near- 
real-time  performance  information  was  fed  back  to  them  in  such  a  way  as  to  close 
the  loop  encompassing  the  subjects  and  the  stimulus.  With  the  emergence  of  new 
sciences  such  as  Artificial  Neural  Systems  and  Chaotic  theory,  the  possibility  of 
achieving  a  rudimentary  form  of  automatic  cognitive  state  estimation  or  "Cognitive 
Mode  Mapping"  has  presented  itself.  Using  these  powerful  analysis  tools,  the 
authors  are  developing  a  system  that  analyzes  and  classifies  EEC  data  from  four 
sites  of  a  subject’s  brain.  The  subjects  produce  this  data  while  performing  five 
selected  cognitive  tasks.  The  objective  of  the  Cognitive  Mode  Mapping  system  is 
_to  identify  the  tasks  based  on  salient  features  embedded  in  the  raw  EEC  signals. 
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Abstract 

For  many  years,  researchers  have  focussed  on 
utilizing  physiological  measures  as  an  indicator  of 
mental  workload.  Research  involving  transient  and 
steady  state  evoked  potential  electroencephalograph 
(EEG)  signals  has  provided  the  foundation  for  further 
study  in  mental  state  estimation.  Studies  involving  sum- 
of-sines  steady  state  evoked  potentials  have  demonstrated 
a  correlation  between  spectral  changes  and  changing 
cognitive  workload.  In  addition,  it  has  been  found  that 
subjects  can  learn  to  control  their  responses  to  steady 
state  visual  stimulus,  provided  near-real-time 
performance  infoimation  was  fed  back  to  them  in  such  a 
way  as  to  effectively  close  the  loop  encompassing  the 
subjects  and  the  stimulus.  With  the  recent  emergence  of 
new  sciences  such  as  Artificial  Neural  Systems  and 
Chaotic  theory,  the  possibility  of  achieving  a 
rudimentary  form  of  automatic  cognitive  state 
estimation  or  “Cognitive  Mode  Mapping ’’  has  presented 
itself.  By  utilizing  these  powerful  analysis  tools,  the 
authors  are  developing  a  system  that  analyzes  and 
classifies  EEG  data  recorded  from  four  sites  of  a 
subject’s  brain.  The  subjects  produce  this  EEG  data 
while  performing  five  selected  cognitive  tasks.  The 
objective  of  the  Cognitive  Mode  Mapping  system  is  to 
identify  these  tasks  based  on  the  salient  features 
embedded  in  the  raw  EEG  signals.  Also,  due  to  the 
fiffinanding  requirements  of  some  environments  (such  as 
jet  fighter  cockpits),  achieving  the  state  recognition  in 
near  real-time  is  critical. 


Translational  effects  served  as  a  formidable  road  block 
to  the  cognitive  mode  mapping  effons,  until  it  was 
discovered  that  chaos  might  provide  a  unique  solution  to 
the  problem.  Phase  portraits  (or  attractors)  were  able 
to  generate  patterns  from  EEG  data  that  were  not 
susceptible  to  time  translation.  It  was  also  discovered 
that  attractor  shapes  generated  from  raw  EEG  data 
changed  with  varying  mental  workload.  And  finally, 
the  patterns  could  easily  be  numerically  vectorized  for 
presentation  to  a  neural  network.  It  was  hypothesized 
that  by  combining  the  phase  space  reconstruction  with 
the  neural  network  algorithms,  an  automatic  free 
running  EEG  feature  recognition  system  could  be 
developed.  It  was  additionally  hypothesized  that  unique 
and  salient  features  could  be  generated  by  performing 
specific  selected  cognitive  tasks,  and  in  this  way  the 
system  might  be  able  to  recognize  these  tasks  based  on 
the  recorded  EEG. 

Experimental  Methodology 

The  development  of  the  experimental  setup  was 
divided  into  three  phases.  The  first  phase  focuses  on 
developing  the  specialized  hardware  and  software 
required  for  proper  acquisition  and  handling  of  the 
subject  data.  Since  the  analysis  procedure  pivots  on  the 
phase  space  reconstruction,  the  EEG  amplifiers  have  to 
be  designed  in  such  a  way  as  to  not  tnnoduce  any  phase 
distortions.  This  means  that  all  the  filters  used  in  the 
amplifiers  have  to  be  phase  linear  over  the  range  of 
operation.  In  this  way  phase  information  contained  in 
the  EEG  is  preserved. 


Introduction 

The  authors  began  research  into  physiological 
Neural  Network  applications  in  1986.  Initial  studies 
were  conducted  utilizing  software  neural  network 
algorithms  in  the  automatic  identification  and 
classification  of  EEG  signals.  In  these  studies,  networks 
performed  panem  recognition  tasks  for  transient  evoked 
potentials,  particularly  for  recognizing  the  presence  or 
absence  of  a  P-300  endogenous  response  to  an  auditory 
tone.  The  performance  of  the  networks  was  very 
promising  however,  the  signals  being  processed  were 
time  synchronized  EEG.  Networks  trained  over 
asynchronous  data  failed  to  perform  with  any  degree  of 
reliability.  It  was  apparent  that  the  neural  networks 
were  extremely  sensitive  to  time  translational  effeas  of 
the  signals.  In  order  to  properly  identify  free  running 
EEG  signals,  these  effects  would  have  to  be  removed. 


The  second  phase,  and  perhaps  the  most  critical, 
involves  the  development  of  software  designed  to 
reconstruct  and  vectorize  the  phase  space  of  the  EEG 
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Phase  space  reconstruction,  part  of  chaos  theory, 
provides  a  powerful  tool  for  preprocessing  of  EEG  data 
in  preparation  for  presentation  to  a  neural  network. 
This  preprocessing  removes  the  effects  of  time 
translation  while  retaining  amplitude,  phase  and 
frequency  infonnation.  Further,  it  can  be  performed  in 


near  real-time  unlike  Discrete  Fourier  Transforms.  It 
has  been  determined  empirically  that  an  n-dimensional 
phase  portrait  of  an  EEC  signal  can  generate  an  input 
vector  that  can  be  easily  utilized  by  neural  network 
paradigms. 

The  third  phase  consists  of  the  experimental  data 
collection  and  subsequent  analysis.  The  experiment 
consists  of  collecting  EEG  data  using  the  multi-site  EEG 
amplifiers  from  four  cranial  sites  (01-02,  P1-P2,  Cl- 
C2,  F1-F2)  while  the  subjects  perform  selected 
cognitive  tasks.  EEG  data  collected  from  each  of  the 
four  electrode  pairs  are  used  to  generate  phase-space 
portraits  (attractors)  whose  digitized  patterns  are 
incorporated  into  a  training  set  and  a  test  set  for  input  to 
a  neural  network.  A  fifth  channel  contains  information 
about  the  task,  specifically  which  classification  category 
it  belongs  to.  Once  the  data  for  a  given  subject  has  been 
collected,  the  neural  network  is  trained  on  the  data  off¬ 
line.  Once  training  is  complete,  the  network 
performance  is  evaluated  by  using  the  test  set  data 
recorded  by  the  same  subject.  Further,  the  trained 
neural  network  is  evaluated  original  subject  is  re¬ 
evaluated  performing  similar  tasks  while  having  their 
mental  workload  monitored  by  the  neural  network  in 
real-time. 
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In  addition,  software  will  be  developed  for 
implementation  of  a  unique  type  of  neural  network 
algorithm  that  changes  in  size  (number  of  computational 
elements  and  their  arrangements)  while  it  trains.  By 
altering  its  architecture  during  the  learning  phase,  the 
network  will  automatically  optimize  its  struemre  to  meet 
the  classification  requirements  of  the  input  data  set. 

Preliminary  Results 


and  the  attractor  consisted  of  1200  consecutive  points. 
The  former  was  chosen  in  respect  to  the  Nyquist 
sampling  criteria,  and  the  latter  was  chosen  arbitrarily. 
After  the  sampling  was  complete,  a  box  counting 
algorithm  was  employed  to  quantize  and  vectorize  the 
data  from  the  reconstructed  phase  space  (attractor)  of 
the  EEG  data.  This  information  was  stored  in  a  disk  file 
along  with  the  associated  tasks  type.  A  total  of  32  such 
vectors  were  generated  and  filed  (16  for  the  training  set 
and  16  for  the  test  set).  Next,  a  backpropagation  neural 
network  algorithm  was  trained  for  approximately  1.5 
hours,  until  the  mean  square  error  for  the  classification 
was  reduced  to  acceptable  levels.  After  network 
training,  the  weight  vectors  that  had  been  iteratively 
modifled  were  stored.  A  second  network  pre-loaded 
with  the  fully  trained  weights  was  then  used  to  analyze 
the  test  set.  For  this  simple  cognitive  state  classification 
problem,  the  network  scored  100%  correct.  That  is  to 
say  that  the  network  by  way  of  the  attractor  scheme, 
could  correctly  identify  the  subjects  assigned  tasks 
directly  from  the  raw  EEG  signal  obtained  from  the 
occipital  lobe  of  the  subject.  After  initial  training  and 
evaluation,  the  subject  was  recalled  and  connected  to  the 
system  in  the  same  way  as  before  except  that  now.  the 
fully  trained  neural  network  was  configured  to  analyze 
the  attractor  box  count  vectors  as  they  were  being 
generated.  In  this  way,  we  were  able  to  test  the  near 
real-time  capabilities  of  the  device.  It  was  discovered, 
that  the  network  was  able  to  correctly  classify  the  EEG 
signals  from  the  subjects  100%  of  the  time.  The 
classification  delay  is  approximately  IS  seconds  due  to 
the  initial  10  seconds  of  data  gathering  and  5  seconds  of 
network  feedfoward  processing  delay.  It  was  also  found 
that  the  trained  network  could  recognize  the  subjects 
EEG  days  after  the  initial  training  took  place.  The 
initial  findings  of  the  study  are  exciting  and  extremely 
promising,  currently  the  authors  are  pursuing  the 
extended  study  utilizing  the  multi-site  data  gathering  and 
a  mote  realistic  complex  cognitive  task  set. 
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